Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix][Frontend] Raise exception when file-like chat template fails to be opened #4292

Merged
merged 4 commits into from
Apr 23, 2024

Conversation

DarkLight1337
Copy link
Member

Currently, when the value provided to --chat-template looks like a file path, but the file fails to be opened, the filename is read as the literal chat template, resulting in much confusion as to why the model fails to generate output.

This PR adds a check for this case so that the user is notified of the error instead of silently using garbage input, which IMO is more in line with expectations.

@simon-mo simon-mo enabled auto-merge (squash) April 23, 2024 17:21
@simon-mo simon-mo merged commit 1e8f425 into vllm-project:main Apr 23, 2024
46 of 47 checks passed
@DarkLight1337 DarkLight1337 deleted the openai-chat-template branch April 24, 2024 01:12
xjpang pushed a commit to xjpang/vllm that referenced this pull request Apr 25, 2024
robertgshaw2-neuralmagic pushed a commit to neuralmagic/nm-vllm that referenced this pull request Apr 26, 2024
alexeykondrat pushed a commit to alexeykondrat/ci-vllm that referenced this pull request May 1, 2024
z103cb pushed a commit to z103cb/opendatahub_vllm that referenced this pull request May 7, 2024
Temirulan pushed a commit to Temirulan/vllm-whisper that referenced this pull request Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants